Latent Variables, State Spaces, and Mixing

We argue that many models for multivariate longitudinal and cross-sectional data analysis have a common ancestry. They all are based on the qualitative idea that if we knew the actual state of the world, the relations between the observed quantities would be truly simple. This is shown to lead directly to factor analysis, IRT, state space models, mixture densities, latent Markov chains, MIMIC, LISREL, and various other common models and technique. It provides a convenient framework for looking at these models. With such a framework often comes a \natural" class of algorithms. For the mixture approach to MVA it is the EM algorithm. 1. Introduction Our starting point in this paper is that we want to describe the relationships between (possibly many) variables, and we want to describe this relationship in simple terms. We look for simplicity, not necessarily because we believe the world is simple, but because simple relationships are easier to manipulate and communicate. We do not deene what we mean by \simplicity", and we do not deene what we mean by \variables". This we leave to the philosophers, who also have to make a living. For Quetelet, Galton, and the early Karl Pearson, the normal distribution was simple. When Pearson 5] rst came across non-normal variation in his biometric work, he tried to maintain this notion of simplicity by assuming that the sample came from a mixture of normal distributions. Thus normality was still the norm, but unfortunately the sample was impure, because it consisted of a mixture of types. If we could have separated the types by observation, we would have seen the normality, but because we couldn't the statistical analysis has to do the job instead. In the same way, the Pearson polychoric model is